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Abstract:  We  consider  the  general  Phase  I  linear  programming  problem  with  a 
convexity  constraint  which  can  be  written  after  some  algebraic  manipulation  in  the 
form: 


Find  xj  >  0,  =  0,  =  1 

where  Pj  are  m-vectors  satisfying  \\Pj\\  =  1.  If  feasible,  von  Neumann’s  Center  of 
Gravity  Algorithm  generates  a  sequence  <  =  1,2,...  of  approximate  solutions 
YjPj^j  —  b1,  =  1,  ij-  >  0  which  converges  in  the  limit  as  <  — »oo  to  a  feasible 

solution  to  the  Phase  I  problem.  We  assume  that  all  perturbed  problems  PjXj  =  6, 

A 

Y,Xj  =  1,  Xj  >  0  are  feasible  for  all  ||6jj  <  r  where  r  >  0  is  given.  We  apply  this  algorithm 
to  m  + 1  perturbed  problems  with  right  hand  sides  b  =  6‘,  i  =  l,2,...,m  + 1  to  obtain  an 
exact  solution  to  the  unperturbed  problem  with  6  =  0  in  T  <  4r"2(m  + 1)3  iterations. 
Each  iteration  consists  of  m(n  +  3)6  multiplications  and  additions  where  6  is  the  non¬ 
zero  coefficient  density. 

Von  Neumann*  in  1948  proposed  the  first  interior  algorithm  for  solving  a  general 
Phase  I  linear  program  with  a  convexity  constraint.  We  will  reproduce  his  proof  that  in 
<  <  1/p2  iterations  an  approximate  solution  Y,Pjxtj  =  6*  will  be  generated  with  |6*||  <  p. 
When  applied  to  a  perturbed  problem  6  =  6  ^  0,  we  will  show  that  in  <  <  4/p2  iterations 
an  approximate  solution  will  be  generated  with  1 6 1  —  6||  <  p. 


verbal  communication 


Geometrically,  in  the  m-space  of  the  columns,  since  ||PJ  =  1,  all  points  Pj  lie  on 
the  surface  of  the  m-dimensional  hypersphere  S0  of  unit  radius  with  center  at  the  origin. 
We  are  given  r  the  radius  of  a  concentric  hypersphere  Sx  C  S0  centered  at  the  origin 
that  lies  in  the  convex  hull  of  the  points  Pj.  Thus  r  is  a  measure  of  how  deeply  the 
origin  is  embedded  in  the  set  of  b  such  that  b  =  £  Pjxj,  xj  >  0,  £  Xj  =  1  is  feasible. 


To  generate  the  m  +  1  different  finite  sequences  whose  bl  approach  m  +  1 

A  .  A  . 

different  points  6‘,  the  b '  are  prechosen.  These  can  be  the  vertices  of  any  simplex  lying 

A  . 

in  the  set  of  feasible  b  that  contains  the  origin  as  an  interior  point.  We  choose  6*  to  be 
the  vertices  of  an  (m  +  1)  equilateral  simplex  whose  center  is  the  origin  and  whose 
vertices  are  located  at  distances  r  •  m/(m  + 1)  from  the  origin;  for  example  the 


coordinates  of  b 

'  may 

be  chosen  j 

as  follows: 
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Figure  1.  The  Iterations  Converge  to  b  Instead  of  the  Origin  0. 
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When  the  ith  sequence  (x*,6‘)  (which  is  converging  towards  S')  reaches  a  point 
b*  =  b*  such  that  Hi*  —  S'J  <  r/(m  + 1),  the  sequence  for  that  i  is  terminated.  Note  that 
all  interior  points  of  Ball,-  of  radius  p  =  r/(m  + 1)  centered  at  S'  lie  inside  the 
hypersphere  Sl  C  SQ.  We  will  show  b*  =  S'  C  Ball,-  is  attainable  by  the  iterative  process. 
Associated  with  S'  is  the  approximate  solution  x'  =  x*  that  generated  it.  Thus  an  upper 
bound  to  generate  all  m  +  1  approximate  solutions  (x',S')  whose  S'  lie  strictly  in  m  + 1 
p-balls  centered  at  S'  can  be  done  in 

(2)  iteration  count  <  4 (m  -f 1)1  p2  =  4 (m  +  l)3/r2  ,  p  =  r/(m  + 1), 

iterations.  The  final  step  is  to  generate  the  feasible  solution  x  to  the  Phase  I  problem  by 
finding  weights  A;  >  0,  x  =  £A,x'>0,  £xj  =  l,  1,  £  PjXj  =  0.  These  weights 
A  =  (Ai, A2,..., Am+1)  are  found  by  solving  the  (m  + 1) x (m  + 1)  system 

(3)  ES‘A,  =  0 

EA;  =  i. 

We  will  prove  that  this  system  has  a  unique  solution  A  =  (Ai,.-->Am+i)  >  0. 

We  now  describe  the  detailed  steps  of  von  Neumann’s  algorithm  for  finding  an 

A 

approximate  solution  to  a  perturbed  problem  E  Pjxj  =  b,  Exj  =  x  >  0  and  give  a 
proof  of  the  rate  of  convergence  of  the  i-th  sequence  to  some  S  =  S'  C  B{.  We  initiate  the 
sequence  of  iterations  by  x  =  x1  =  (1,0,..., 0),  bl  =  Pv  Inductively  let  xt_1,  bl~l  be  the 
t  —  1  approximation.  We  use  it  to  generate  x‘,  bl. 
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A 

Referring  to  Figure  2,  Pt  is  selected  as  that  Pj  such  that  Pj  —  b  makes  the  sharpest 

A  - 

angle  8  with  direction  b  —  6  ,  namely 

(4)  s  =  ARGMAX  |[S-4‘-1JT[Pi-6]/|Pi-&||. 

which  can  be  carried  out  in  m(n  +  3)  operations  assuming  [Pj  —  6|  is  preprocessed.  The 
triangle  ft*-1,  Pt,  6  will  be  labeled  ABC.  The  next  approximation  point  H  =  b*  is  the 
foot  of  perpendicular  dropped  from  C  onto  the  side  AB  of  the  triangle  ABC.  From  the 
figure,  it  is  clear  that  H  is  a  weighted  convex  combination  of  A  and  B  with  weights 
proportional  to  cos  92  and  cos  9U  i.e., 

(5)  6‘  =  (cos  92  •  6t-1  +  cos  6t  •  P,)/( cos  82  +  cos  flj), 
xl  =  (cos  92  •  xt_1  +  cos  8l  ■  Ut)/(cos  d2  +  cos  02), 


where  U,  is  the  unit  n  vector  with  1  in  component  s.  cos  01  and  cos  82  are  computed  by 

lt-1  D  \  t  D  ft  1.1-1  A 


(6) 


C0S  6,2  "HS-Pjr-1 -/>,«’  C°S^  ||P.-6‘-1||!|6-6*-ir 


In  order  to  determine  the  rate  of  convergence,  note  8  <  tt/2  because  if,  on  the 

A 

contrary,  9  >  7r/2  then  all  points  Pj  would  lie  on  one  side  of  the  hyperplane  through  b 
orthogonal  to  bl~l  -  b  implying  that  6  =  6*  for  the  i-th  sequence  lies  outside  the  convex 
hull  of  the  Pj's  contrary  to  our  assumption  that  all  points  located  at  a  distance  r  or  less 

a  , 

from  the  origin  are  in  the  set  of  feasible  6  (i.e.,  6*  by  construction  lies  in  the  interior  of 
the  set  of  feasible  6  C  SY  at  a  distance  r/(m  + 1)  from  the  boundary  of  Sv  To  simplify 
the  notation,  let 


A,.,  =  r-,-S|andA,  =  |6,-S|, 


then 

(7)  A,  =  At_!  sin  ^and  Ae  =  ||P,  -  6||  sin  92. 

Therefore,  noting  91+92  =  9  <  tt/2, 

2  .2 

+( — . -  |  =  sin2  9j  +sin2  82  <  1. 

A 

Recalling  that  diameter  of  the  hypersphere  is  2,  it  follows  that  \\P,  —  6||  <2  and 
therefore  for  r  =  2, 3, . . . ,  t: 


Comment:  These  inequalities  can  be  made  tighter  when  6  =  0  because  |P,  —  S|  = 
\P.l  =  1  If  so,  (8)  can  be  replaced  by  (AT/Ar_1)2  +  A2  <  1  and  the  development  that 
follows  can  be  modified  accordingly  with  the  conclusion  that  if  the  von  Neumann 
iterative  process  is  applied  to  the  case  6  =  0  instead  of  to  6*  ^  0  an  approximation  6‘ 
such  that  1)6*11  <  p  can  be  attained  in  less  that  1/p2  iterations  (instead  of  less  than  4/p2 
iterations). 

Dividing  (8)  through  by  (Ar)2  for  r  =  2, 

(1/A,-!)2  +  (1/4)  <  (1/At)2 
(1/A,_2)2  +  (1/4)  <  (1/At_,)2 

(1/A)2  +  (1/4)  <  (1/A2)2. 

Summing  the  above,  canceling  terms  common  to  both  sides  of  the  sum  and,  recalling 
Aj  <  2,  we  have 

(10)  (1/A,)2  >  (1/4)  +  (<  — 1)/4  =  t/4. 

We  conclude  that  t  <  4/A2  iterations,  i.e.  less  than  4/p2  iterations  would  be  needed  for 
the  ith  sequence  to  terminate  by  reaching  6*  =  6',  an  interior  point  of  the  p-ball  centered 

A  , 

at  b\  Since  p  =  r/(m  + 1)  and  there  are  (m  -f  1)  p-balls,  the  upper  bound  on 

(11)  iteration  count  <  4(m  +  l)3/r2. 

What  remains  to  show  is  that  the  (m  + 1)  x  (m  + 1)  system  (3)  can  be  solved,  that  the 
solution  A  is  unique,  and  that  A  =  (Al5  A2, . . . ,  Am+i)  >  0. 

Existence  of  Separating  Hyperplanes:  Let  y  =  represent  a  general 

point  in  Rm.  The  equation  of  any  hyperplane  through  the  origin  has  the  form  ary  =  0. 
This  hyperplane  is  said  to  separate  y 1  from  y2  if  aTy 1  and  ary2  are  of  opposite  signs. 

Fact  1.  Each  hyperplane  (b‘)Ty  =  0  for  i  =  l,2,...,m  separates  any  point  in  the  p- 

A  .  A  ■ 

ball  centered  at  6'  from  any  point  lying  in  any  of  the  other  p-balls  centered  at  b}. 

Proof:  Because  of  the  m  -f  1  fold  symmetry  of  the  equilateral  simplex  it  is  sufficient 
to  demonstrate  that  the  hyperplane  (bm+1)Ty  =  0  separates  6m+1  from  6m  where 
||6m+1  —  6m+1||  <r/(m-f  1)  and  ||6m  — 6m||  <r/(m  +  l).  The  coordinates  of  6m+1  and  6m 
defined  by  (1)  are  6m+1  =  (0, 0, . . . , rm/(ra  +  1))T  and  6m  =  (0,0,...,Hm  —  1/^m  + 1, 
-r/(m  +  l))T.  The  hyperplane  (6m+1)Tr/  =  0  reduces  to  (0,...,l)j/  =  U^y  =  0.  Letting 
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bm+1  =  bm+1  +  «  where  |u|  <  r/(m  + 1),  we  have  UJ*  6m+1  =  6JS+1  =  &™+1  +  itm  > 
rm/(m  + 1)  —  r/(m  + 1)  >  0  since  <  r/(m  + 1).  Letting  bm  =  bm  +  v  where 

H  <  r/(m  + 1),  we  have  =  &J2  +  vm  <  — r/(m  + 1)  +  r/(m  + 1)  =  0.  Thus  UJbm+1 

and  Um  bm  have  opposite  signs  and  so  the  hyperplane  U,^y  =  0  separates  6m+1  from  6m.  I 

The  Separating  Hyperplanes  Theorem  below  states  conditions  which  imply  that  the 
points  61,P,...,6TO+1  are  the  vertices  of  a  simplex  containing  the  origin  in  its  interior. 
That  these  conditions  are  satisfied  follows  from  Fact  1. 

Separating  Hyperplanes  Theorem:  Given  (1)  that  (S1,S2,...,Sm+1)  are  any  (m  +  1) 
vertices  of  an  m-dimensional  simplex  T  containing  the  origin;  given  (2)  that  a'y  =  0  for 

A  .  A  . 

i  =  1,2,..., m  +  l  are  the  equations  of  m  +  1  hyperplanes  separating  6*  from  b3  for  all 
j  -£i\  and  given  (3)  any  m  +  1  points  61,62,...,6m+1  such  that  each  hyperplane  a'y  =  0 
separates  b'  (on  the  same  side  as  b')  from  b3  for  all  j  /  i;  then  51,52,...6m  are  the 
vertices  T  of  an  m-dimensional  simplex  that  contains  the  origin  as  an  interior  point. 

Proof:  Since  the  simplex  associated  with  T  contains  the  origin,  we  know  there  exist 
A i  >  0,  A,-  >  0  such  that 

(13.1)  £b%+  £b%  =  0 

(13.2)  £Ai+  EA,  =  1. 

Before  continuing  with  the  proof,  we  show  two  more  facts: 

Fact  2.  If  (A,  A)  is  a  feasible  solution  to  (13.1),  (13.2),  then  A,-  +  A;  >  0  for  all  i. 

A  * 

Suppose,  on  the  contrary,  Ajt  =  0,  A*.  =  0  for  some  k.  Multiply  (13.1)  on  the  left  by  cr; 
recall,  by  assumption,  okb3  <  0  and  akV  <  0  for  all  j  ^  k.  We  have 


(14.1) 

E  (a*60Ai  +E  {akb%  =0 

j** 

(14.2) 

E^i  +E^i  ==  1) 

implying,  that  (14.1)  is  the  sum  of  non-negative  terms  (not  all  zero  by  (14.2),  a 
contradiction.  I 

Fact  3.  If  T  is  any  simplex  containing  the  origin  whose  vertices  i  are  separated 
from  the  remaining  vertices  j  ^  i  by  a  hyperplane  a'y  =  0  for  each  i,  then  T  contains 
the  origin  strictly  in  its  interior.  I 
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Fact  3  follows  from  Fact  2  by  setting  b*  =  S'  for  all  i. 


Continuing  with  the  proof  of  the  separating  hyperplanes  theorem,  define  S  and 

Um+ 1  by 


(15) 


A 

A 

0 

b1 

b2  ...  bm+1 

1 

1  1 

i  ^m+1  _ 

0 

- 

- 

1 

Since  T  are  the  vertices  of  an  m-dimensional  simplex  by  assumption,  it  means  that  S  is 
non-singular  and  that  SBA  =  Um+1  can  be  solved  for  A  and,  when  solved,  A  >  0.  From 
Fact  3  it  follows  that  A  >  0.  We  view  *35  as  a  feasible  non-degenerate  basis  and  consider 


in  the  basis  because, 


as  an  incoming  non-basic  column.  We  assert  it  will  replace  j 

on  the  contrary,  if  it  replaced  some  column  k  /  1  in  the  basis,  it  would  imply  after  the 
replacement  that  both  Afc  and  Atare  0  in  a  feasible  solution,  contrary  to  Fact  2.  By 

Pi 


replacing  in  turn  basis  columns 


by 


by 


1 


,  etc.,  we  arrive  at  the 


conclusion  that  T  are  the  vertices  of  a  simplex  containing  the  origin.  It  then  follows 
from  Fact  3  that  this  simplex  contains  the  origin  as  a  strictly  interior  point.  I 


This  completes  the  proof  that  the  (m  +  1)  sequences  converge  to  m  +  1  points  6'  in 
less  than  4 (m  -f  1  )3/r2  iterations.  By  applying  the  weights  A,-  >  0  to  the  corresponding  x\ 
we  generate  the  exact  solution  x  to  the  Phase  I  linear  program. 


One  final  remark:  Just  because  an  algorithm  is  polynomial  does  not  necessarily 
make  it  practical.  The  von  Neumann  algorithm  has  a  poor  convergence  rate.  Like  the 
simplex  method  each  of  its  iterations  requires  about  mn6  multiplications  and  additions 
where  S  is  the  density  of  non-zero  coefficients.  When  applied  to  (m  +  1)  perturbed 
problems  as  we  do  in  this  paper,  we  obtain  an  upper  bound  of  4(m  +  l)3/r2  iterations 
where  0  <  r  <  1.  The  moral  of  this  tale  is  that,  like  gunners,  we  may  do  better  by  first 
bracketing  the  target  and  then  applying  a  final  correction. 
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